Panoptic图像分割是计算机视觉任务，即在图像中查找像素组并为其分配语义类别和对象实例标识符。由于其在机器人技术和自动驾驶中的关键应用，图像细分的研究变得越来越流行。因此，研究社区依靠公开可用的基准数据集来推进计算机视觉中的最新技术。但是，由于将图像标记为高昂的成本，因此缺乏适合全景分割的公开地面真相标签。高标签成本还使得将现有数据集扩展到视频域和多相机设置是一项挑战。因此，我们介绍了Waymo Open DataSet：全景视频全景分割数据集，这是一个大型数据集，它提供了用于自主驾驶的高质量的全景分割标签。我们使用公开的Waymo打开数据集生成数据集，利用各种相机图像集。随着时间的推移，我们的标签是一致的，用于视频处理，并且在车辆上安装的多个摄像头保持一致，以了解全景的理解。具体而言，我们为28个语义类别和2,860个时间序列提供标签，这些标签由在三个不同地理位置驾驶的自动驾驶汽车上安装的五个摄像机捕获，从而导致总共标记为100k标记的相机图像。据我们所知，这使我们的数据集比现有的数据集大量数据集大的数量级。我们进一步提出了一个新的基准，用于全景视频全景分割，并根据DeepLab模型家族建立许多强大的基准。我们将公开制作基准和代码。在https://waymo.com/open上找到数据集。

图像预训练，当前用于广泛视觉任务的当前事实范式在视频识别领域中通常不太受青睐。相比之下，一种共同的策略是直接从头开始使用时空卷积神经网络（CNN）训练。尽管如此，有趣的是，通过仔细研究这些从划痕学到的CNN，我们注意到存在某些3D内核比其他人具有更强的外观建模能力，可以说表明外观信息在学习中已经很好地散布了。受到这一观察的启发，我们假设有效利用图像预训练的关键在于学习空间和时间特征的分解，并将图像预训练作为初始化3D内核之前的外观。此外，我们提出了空间可分离（STS）卷积，该卷积将特征通道明确将特征通道分为空间和时间基团，以进一步使时空特征更彻底地分解3D CNN。我们的实验表明，简单地用ST替换3D卷积可以显着改善3D CNN的范围，而无需增加参数和计算动力学400和一些v2的计算。此外，这条新的培训管道始终以显着加速的视频识别取得更好的结果。例如，在强大的256- epecoch 128-GPU基线上，我们在Kinetics-400上获得了 +0.6％的慢速1，同时仅以40个GPU进行微调，而对50个时代进行了微调。代码和型号可在https://github.com/ucsc-vlaa/image-pretraining-for-video上找到。

变压器出现为可视识别的强大工具。除了在广泛的视觉基准上展示竞争性能外，最近的作品还争辩说，变形金刚比卷曲神经网络（CNNS）更强大。令人惊讶的是，我们发现这些结论是从不公平的实验设置中得出的，其中变压器和CNN在不同的尺度上比较，并用不同的训练框架应用。在本文中，我们的目标是在变压器和CNN之间提供第一个公平和深入的比较，重点是鲁棒性评估。通过我们的统一培训设置，我们首先挑战以前的信念，使得在衡量对抗性鲁棒性时越来越多的CNN。更令人惊讶的是，如果他们合理地采用变形金刚的培训食谱，我们发现CNNS可以很容易地作为捍卫对抗性攻击的变形金刚。在关于推广样本的泛化的同时，我们显示了对（外部）大规模数据集的预训练不是对实现变压器来实现比CNN更好的性能的根本请求。此外，我们的消融表明，这种更强大的概括主要受到变压器的自我关注架构本身的影响，而不是通过其他培训设置。我们希望这项工作可以帮助社区更好地理解和基准变压器和CNN的鲁棒性。代码和模型在https://github.com/ytongbai/vits-vs-cnns上公开使用。

The Coronavirus disease 2019 (COVID-19) was first identified in Wuhan, China, in early December 2019 and now becoming a pandemic. When COVID-19 patients undergo radiography examination, radiologists can observe the present of radiographic abnormalities from their chest X-ray (CXR) images. In this study, a deep convolutional neural network (CNN) model was proposed to aid radiologists in diagnosing COVID-19 patients. First, this work conducted a comparative study on the performance of modified VGG-16, ResNet-50 and DenseNet-121 to classify CXR images into normal, COVID-19 and viral pneumonia. Then, the impact of image augmentation on the classification results was evaluated. The publicly available COVID-19 Radiography Database was used throughout this study. After comparison, ResNet-50 achieved the highest accuracy with 95.88%. Next, after training ResNet-50 with rotation, translation, horizontal flip, intensity shift and zoom augmented dataset, the accuracy dropped to 80.95%. Furthermore, an ablation study on the effect of image augmentation on the classification results found that the combinations of rotation and intensity shift augmentation methods obtained an accuracy higher than baseline, which is 96.14%. Finally, ResNet-50 with rotation and intensity shift augmentations performed the best and was proposed as the final classification model in this work. These findings demonstrated that the proposed classification model can provide a promising result for COVID-19 diagnosis.

Users' physical safety is an increasing concern as the market for intelligent systems continues to grow, where unconstrained systems may recommend users dangerous actions that can lead to serious injury. Covertly unsafe text, language that contains actionable physical harm, but requires further reasoning to identify such harm, is an area of particular interest, as such texts may arise from everyday scenarios and are challenging to detect as harmful. Qualifying the knowledge required to reason about the safety of various texts and providing human-interpretable rationales can shed light on the risk of systems to specific user groups, helping both stakeholders manage the risks of their systems and policymakers to provide concrete safeguards for consumer safety. We propose FARM, a novel framework that leverages external knowledge for trustworthy rationale generation in the context of safety. In particular, FARM foveates on missing knowledge in specific scenarios, retrieves this knowledge with attribution to trustworthy sources, and uses this to both classify the safety of the original text and generate human-interpretable rationales, combining critically important qualities for sensitive domains such as user safety. Furthermore, FARM obtains state-of-the-art results on the SafeText dataset, improving safety classification accuracy by 5.29 points.

Conditional diffusion probabilistic models can model the distribution of natural images and can generate diverse and realistic samples based on given conditions. However, oftentimes their results can be unrealistic with observable color shifts and textures. We believe that this issue results from the divergence between the probabilistic distribution learned by the model and the distribution of natural images. The delicate conditions gradually enlarge the divergence during each sampling timestep. To address this issue, we introduce a new method that brings the predicted samples to the training data manifold using a pretrained unconditional diffusion model. The unconditional model acts as a regularizer and reduces the divergence introduced by the conditional model at each sampling step. We perform comprehensive experiments to demonstrate the effectiveness of our approach on super-resolution, colorization, turbulence removal, and image-deraining tasks. The improvements obtained by our method suggest that the priors can be incorporated as a general plugin for improving conditional diffusion models.

In a high dimensional linear predictive regression where the number of potential predictors can be larger than the sample size, we consider using LASSO, a popular L1-penalized regression method, to estimate the sparse coefficients when many unit root regressors are present. Consistency of LASSO relies on two building blocks: the deviation bound of the cross product of the regressors and the error term, and the restricted eigenvalue of the Gram matrix of the regressors. In our setting where unit root regressors are driven by temporal dependent non-Gaussian innovations, we establish original probabilistic bounds for these two building blocks. The bounds imply that the rates of convergence of LASSO are different from those in the familiar cross sectional case. In practical applications given a mixture of stationary and nonstationary predictors, asymptotic guarantee of LASSO is preserved if all predictors are scale-standardized. In an empirical example of forecasting the unemployment rate with many macroeconomic time series, strong performance is delivered by LASSO when the initial specification is guided by macroeconomic domain expertise.

Existing methods for large-scale point cloud semantic segmentation require expensive, tedious and error-prone manual point-wise annotations. Intuitively, weakly supervised training is a direct solution to reduce the cost of labeling. However, for weakly supervised large-scale point cloud semantic segmentation, too few annotations will inevitably lead to ineffective learning of network. We propose an effective weakly supervised method containing two components to solve the above problem. Firstly, we construct a pretext task, \textit{i.e.,} point cloud colorization, with a self-supervised learning to transfer the learned prior knowledge from a large amount of unlabeled point cloud to a weakly supervised network. In this way, the representation capability of the weakly supervised network can be improved by the guidance from a heterogeneous task. Besides, to generate pseudo label for unlabeled data, a sparse label propagation mechanism is proposed with the help of generated class prototypes, which is used to measure the classification confidence of unlabeled point. Our method is evaluated on large-scale point cloud datasets with different scenarios including indoor and outdoor. The experimental results show the large gain against existing weakly supervised and comparable results to fully supervised methods\footnote{Code based on mindspore: https://github.com/dmcv-ecnu/MindSpore\_ModelZoo/tree/main/WS3\_MindSpore}.

Storytelling and narrative are fundamental to human experience, intertwined with our social and cultural engagement. As such, researchers have long attempted to create systems that can generate stories automatically. In recent years, powered by deep learning and massive data resources, automatic story generation has shown significant advances. However, considerable challenges, like the need for global coherence in generated stories, still hamper generative models from reaching the same storytelling ability as human narrators. To tackle these challenges, many studies seek to inject structured knowledge into the generation process, which is referred to as structure knowledge-enhanced story generation. Incorporating external knowledge can enhance the logical coherence among story events, achieve better knowledge grounding, and alleviate over-generalization and repetition problems in stories. This survey provides the latest and comprehensive review of this research field: (i) we present a systematical taxonomy regarding how existing methods integrate structured knowledge into story generation; (ii) we summarize involved story corpora, structured knowledge datasets, and evaluation metrics; (iii) we give multidimensional insights into the challenges of knowledge-enhanced story generation and cast light on promising directions for future study.

Establishing open and general benchmarks has been a critical driving force behind the success of modern machine learning techniques. As machine learning is being applied to broader domains and tasks, there is a need to establish richer and more diverse benchmarks to better reflect the reality of the application scenarios. Graph learning is an emerging field of machine learning that urgently needs more and better benchmarks. To accommodate the need, we introduce Graph Learning Indexer (GLI), a benchmark curation platform for graph learning. In comparison to existing graph learning benchmark libraries, GLI highlights two novel design objectives. First, GLI is designed to incentivize \emph{dataset contributors}. In particular, we incorporate various measures to minimize the effort of contributing and maintaining a dataset, increase the usability of the contributed dataset, as well as encourage attributions to different contributors of the dataset. Second, GLI is designed to curate a knowledge base, instead of a plain collection, of benchmark datasets. We use multiple sources of meta information to augment the benchmark datasets with \emph{rich characteristics}, so that they can be easily selected and used in downstream research or development. The source code of GLI is available at \url{https://github.com/Graph-Learning-Benchmarks/gli}.